Papers with discriminative and generative tasks
It’s not Rocket Science: Interpreting Figurative Language in Narratives (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing text representations by design rely on compositionality, while figurative language is often non-compositional. |
| Approach: | They propose to use a pre-trained language model to interpret figurative language types to adopt human strategies for interpreting figurativ language types: inferring meaning from context and relying on constituent words’ literal meanings. |
| Outcome: | The proposed models perform significantly worse than humans on discriminative and generative tasks, bridging the gap from human performance. |
Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing Turkish benchmarks lack task diversity or culturally relevant content . Cetvel combines a broad range of discriminative and generative tasks . |
| Approach: | They propose a benchmark to evaluate large language models in Turkish . Cetvel combines a broad range of discriminative and generative tasks . they find that Turkish-centric instruction-tuned models generally underperform . |
| Outcome: | The proposed benchmark covers 23 tasks grouped into seven categories . it shows that Turkish-centric instruction-tuned models underperform relative to multilingual or general-purpose models despite being tailored for the language. |
PAGnol: An Extra-Large French Generative Model (2022.lrec-1)
Copied to clipboard
Julien Launay, E.l. Tommasone, Baptiste Pannier, François Boniface, Amélie Chatelain, Alessandro Cappelli, Iacopo Poli, Djamé Seddah
| Challenge: | a growing number of pre-trained language models are available in many different languages. |
| Approach: | They propose a French-language GPT model with scaling laws to train it efficiently . they evaluate the models on discriminative and generative tasks in French . |
| Outcome: | The proposed model trains with the same computational budget as CamemBERT, a model 13 times smaller. |